Hello


The burden of malaria

Intro


How to eliminate

Intro


Challenges in evaluation

Intro


Why evaluate

Intro


How to evaluate

Intro


What are we evaluating

Intro


The data

Methods


Identification strategy and variables

Methods

Rainfall

Methods


(Pretty similar everywhere)

Temperature

Methods


(Pretty similar everywhere)

Difference in differences

Methods


Why we can’t use difference in differences

Methods


Synthetic control model

Methods


Magude vs. other districts

Results


Incidence, not adjusted for weighted contributions.

Magude vs. other districts by age

Results


Effect is clearer in younger children.

0-4 year-olds: Magude vs synthetic control

Results

img/synthetic1.png

5+ year-olds: Magude vs synthetic control

Results

img/synthetic2.png

Placebo testing for diarrhea

Results

img/diarrhea.png

Effect of interevention

Results

img/ranjeeta_table.png


Relative to the synthetic control, the average treatment effect during the first post-intervention malaria season was a reduction in weekly incidence by 4 and 2 cases per 1,000 in those aged 0-4 and 5 or older, respectively. During peak malaria season (January - March), 1,939 cases were prevented in 2016, and 3,405 in 2017.

Our findings

Results


Implications

Discussion

Thank you

Fin

http://www.economicsofmalaria.com
joe@databrew.cc

Extra slides

Supplementary slides begin here.

The synthetic control

Results

Indicator Magude 0-4 Synthetic Magude 0-4 Magude 5+ Synthetic Magude 5+
Incidence, rainy season 2015 3.65 5.10 4.02 4.12
Incidence, rainy season 2014 8.87 6.93 8.53 6.29
LLIN coverage 24.82 24.77 24.83 25.22
Pre-intervention precipitation 1.71 1.74 1.70 1.71
Pre-intervention temperature 24.52 24.51 24.52 24.49
Post-intervention precipitation 1.84 1.82 1.84 1.83

Comparison with other findings

Discussion

Quasi-experimental methods in similar contexts

Discussion

Comparison with RCT

Discussion

Scaling up the method?

Discussion

Limitations

Discussion

  1. We rely on surveillance malaria cases reported at the health facility level, which is an incomplete and imperfect proxy for true malaria infection incidence since health-seeking behavior is especially low in low-income countries settings and may be different across geography and time.

  2. The health facility registries from which we gathered data are themselves imperfect; we have no mechanisms by which to validate the entirety of the data. Still, BES data offers the best country-wide comparison of malaria trends at a granular level (weekly and by district).

  3. Even with quasi-experimental data, we still can’t fully control for abnormalities… (see next slide)

2017 has been anything but normal

Discussion

---
title: "Short-term impact of a malaria elimination campaign in Southern Mozambique"
output: 
  flexdashboard::flex_dashboard:
    theme: yeti
    storyboard: true
    social: menu
    source: embed
---

```{r setup, include=FALSE}
library(flexdashboard)
```

```{r, echo = FALSE, warning = FALSE, message = FALSE, comment = NA, error= FALSE, cache = F}
library(knitr)
opts_chunk$set(comment = NA, 
               echo = FALSE, 
               warning = FALSE, 
               message = FALSE, 
               error = TRUE, 
               cache = FALSE)
```


```{r, results = 'hide'}
# Packages
library(ggplot2)
library(cism)
library(rgeos)
library(maptools)
library(rgdal)
library(tidyr)
library(RColorBrewer)
library(dplyr)
library(leaflet)
library(readr)
library(ggthemes)
library(ggrepel)
library(knitr)
library(kableExtra)
options(knitr.table.format = "html") 
# Define a theme
theme_maltem <- ggthemes::theme_hc
```


```{r}
if('presentation_data.RData' %in% dir()){
  load('presentation_data.RData')
} else {
  source('../../master.R', chdir = TRUE)

  tablify <- function(x, n = 5){
    DT::datatable(x,
                  selection = 'none',
                  escape = FALSE,
                  options = list(#sDom  = '<"top">lrt<"bottom">ip',
                                 pageLength = n, 
                                 dom = 'tip'))
  }
  kablify <- function(x, size = 16){
    kable(x) %>% 
    kable_styling(bootstrap_options = c("striped",
    "hover",
    "condensed"),
    font_size = size,
    position = 'float_left')
  }
  save.image('presentation_data.RData')
}

```





### Hello


```{r}
# africa <- cism::africa
# africa_small <- thinnedSpatialPoly(SP = africa,
#                                      minarea = 0,
#                                      tolerance = 20,
#                                      topologyPreserve = TRUE)
# cols <- colorRampPalette(brewer.pal(9, 'Blues')[3:9])(nrow(africa_small))
# cols <- sample(cols)
# cols <- adjustcolor(cols, alpha.f = 0.8)
# plot(africa_small, col = cols, border = NA)

ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'How effective is a malaria\nelimination campaign in Mozambique?'),
            size = 8)
```




***


- Joe Brew  
- joe@databrew.cc  
- [economicsofmalaria.com](http://economicsofmalaria.com) 
- [economicsofmalaria.com/tgh](http://economicsofmalaria.com/tgh) 


###  The burden of malaria

Intro




```{r}
options(scipen = '999')
x <- data_frame(year = c(2000, 2015),
                deaths = c(839000, 438000))
g1 <- 
  ggplot(data = x,
       aes(x = year,
           y = deaths)) +
  geom_area(fill = 'darkorange',
            alpha = 0.6) +
  geom_point(alpha = 0.6,
             size = 2,
             color = 'blue') +
  theme_maltem() +
  labs(x = 'Year',
       y = 'Deaths',
       title = 'How many people die?')

y <- data_frame(region = c('Africa', 'Everywhere else'),
                percent = c(90, 10))
g2 <- 
  ggplot(data = y,
       aes(x = region,
           y = percent)) +
  geom_bar(stat = 'identity',
           fill = 'darkorange', 
           alpha = 0.6) +
  theme_maltem() +
  labs(x = 'Region',
       y = 'Percent',
       title = 'Where do they die?')
Rmisc::multiplot(g1, g2, cols = 2)
```

***

- 2015: 212M cases; 429K deaths (global). 
- Mozambique among ten highest burden countries in world. 
- 29% reduction since 2010.  
- Now, focus on _elimination_ and _eradication_.

###  How to eliminate

Intro


```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = '?'),
            size = 100)
```

***

- Scale up vs. innovate.  
- Mass drug administration (MDA).  
- Other, more time-tested strategies (ITNs, IRS).
- The need for operational research. 

###  Challenges in evaluation

Intro

```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'Evaluation != Easy'),
            size = 13)
```

***

- Difficult to measure impact of interventions in operational research.  
- OR is usually not experimental, not randomized, not controlled.
- Before vs. after and case vs. control approaches are useful, but biased.


###  Why evaluate

Intro


```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'Evaluation = Necessary'),
            size = 13)
```

***

- WHO: "the need for more research on the optimum methods of implementing MDA programs, promoting community participation and compliance with treatment, and evaluating their effectiveness.""

- Sustainability and cost-effectiveness are among the main concerns of funders, stakeholders and policymakers.

- Evidence of the causal impact of malaria elimination initiatives is a pre-requisite for evaluating their cost-effectiveness. 


###  How to evaluate

Intro


```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'We need a\nCOUNTERFACTUAL'),
            size = 13)
```

***

- Since we don't have an experiment, we need quasi-experimental approaches.  
- The synthetic control method (SCM).  
- We want to construct a _realistic_ counterfactual telling us _what would have happened_ without the campaign.  




###  What are we evaluating

Intro


```{r}
cols <- c('darkorange', 'darkgreen')
moz2_fortified <- moz2_fortified
map <- moz2_fortified %>%
  filter(id %in% c('Manhiça',
                   'Magude'))
ggplot() +
  geom_polygon(data = map,
               aes(x = long,
                   y = lat,
                   group = id,
                   fill = id),
               color = 'white',
               # color = 'grey',
               alpha = 0.8) +
  theme_map() +
  scale_fill_manual(name = '',
                    values = cols) +
  # theme_black(base_size = 0) +
  theme(axis.line = element_blank(), axis.text = element_blank(),
        axis.ticks = element_blank(), axis.title = element_blank(),
        legend.position = 'none') 
```

***

- Malaria elimination campaign in Magude.  
- Deployed during rainy seasons of 2015-6 and 2016-7. 
- Consisted of MDA, IRS, and surveillance. 
- ITN coverage was already high from 2014. 


###  The data

Methods



```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  group_by(date, district) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

ggplot(data = x,
       aes(x = date,
           y = p)) +
  geom_line(alpha = 0.8,
            color = 'darkred') +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Weekly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Weekly incidence: all districts',
       subtitle = 'All age groups') +
  theme_maltem() +
  facet_wrap(~district) +
  theme(legend.position="none") +
  theme(strip.background = element_rect(fill="white"),
        strip.text = element_text(color = grey(0.1), size = 6)) +
  theme(axis.text.x = element_text(angle = 90))
```

***

- Mozambique’s Boletim Epidemiológico Semanal (BES) of the NMCP.  
- Weekly clinical cases, by age group, by health facility, for each of a few high risk diseases
, the epidemic disease reporting system used by the NMCP. 
- Population estimates from the National Statistical Institute (INE). 

###  Identification strategy and variables


Methods



```{r}
ggplot() + theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'analysis variables:\n
- LLINs coverage\n    
- IRS coverage\n  
- Average weekly temperature\n
- Weekly rainfall'),
            size = 7)
```



###  Rainfall


Methods


```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(place = factor(place, levels = c('Other districts', 'Magude'))) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, district, place) %>%
  summarise(precipitation = mean(precipitation),
            temp = mean(temp),
            temp_max = max(temp_max),
            temp_min = min(temp_min))

g <- ggplot() +
  geom_line(alpha = 0.2,
            data = x %>% filter(place == 'Other districts'),
       aes(x = date,
           y = precipitation,
           group = district)) +
    geom_line(alpha = 0.8,
              color = 'darkred',
            data = x %>% filter(place == 'Magude'),
       aes(x = date,
           y = precipitation,
           group = district)) +
  labs(x = 'Date',
       y = 'Milimeters',
       title = 'Precipitation: Magude vs. other districts',
       subtitle = 'Monthly average precipitation, Magude in red') +
  theme_maltem()
g 
```

***

(Pretty similar everywhere)


###  Temperature


Methods


```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(place = factor(place, levels = c('Other districts', 'Magude'))) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, district, place) %>%
  summarise(precipitation = mean(precipitation),
            temp = mean(temp),
            temp_max = max(temp_max),
            temp_min = min(temp_min))

ggplot() +
  geom_line(alpha = 0.2,
            data = x %>% filter(place == 'Other districts'),
       aes(x = date,
           y = temp,
           group = district)) +
    geom_line(alpha = 0.8,
              color = 'darkred',
            data = x %>% filter(place == 'Magude'),
       aes(x = date,
           y = temp,
           group = district)) +
  labs(x = 'Date',
       y = 'Degrees (celcius)',
       title = 'Average temperature: Magude vs. other districts',
       subtitle = 'Monthly average, Magude in red') +
  theme_maltem() +
  xlim(as.Date('2014-01-01'),
               max(x$date))
```

***


(Pretty similar everywhere)


###  Difference in differences


Methods


```{r}
a <- seq(1, 5, length = 10)
b<- c(seq(2, 4, length = 5), seq(5, 7, length = 5))
x <- data_frame(time = 1:10,
                 a,
                 b)
x <- tidyr::gather(x, key, value, a:b)
x <- x %>%
  mutate(key = ifelse(key == 'a', 'Control', 'Intervention'))
ggplot(data = x,
       aes(x = time,
           y = value,
           color = key)) +
  geom_line() +
  geom_point() +
  theme_void() +
  scale_color_manual(name = '',
                     values = c('red', 'blue')) +
  geom_vline(xintercept = 5,
             alpha = 0.6,
             lty = 2)
```

***

- We _could_ compare changes in malaria incidence to other districts using a DD model. 
- Often used to estimate treatment effects in such settings by contrasting changes in outcome pre- and -post intervention, for the treatment and control groups.




###  Why we can't use difference in differences


Methods


```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, place) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

g1 <- ggplot(data = x,
       aes(x = date,
           y = p,
           group = place)) +
  geom_line(aes(color = place), alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Monthly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Incidence: Magude vs. other districts',
       subtitle = 'All age groups') +
  theme_maltem() +
  xlim(as.Date('2013-01-01'), max(x$date)) +
  geom_vline(xintercept = as.numeric(as.Date('2015-10-01')))


x2 <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, place) %>%
  summarise(p = sum(cases) / sum(population) * 1000) %>%
  group_by(date) %>%
  summarise(magude_to_other = p[place == 'Magude'] / 
              p[place == 'Other districts'] * 100)

g2 <- ggplot(data = x2,
       aes(x = date,
           y = magude_to_other)) +
  geom_line(alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  geom_hline(yintercept = 100,
             col = 'darkred',
             alpha = 0.6,
             lty = 2) +
  labs(x = 'Date',
       y = 'Percent') +
  labs(title = 'Magude incidence as % of other districts',
       subtitle = 'All age groups') +
  theme_maltem() +
  xlim(as.Date('2013-01-01'), max(x$date))
g1
# Rmisc::multiplot(g1, g2, cols = 2)
```


***

- Magude shows a downward trend in incidence over time, even prior to the elimination campaign.
- Lack of a visible parallel trend between treatment and control groups 


###  Synthetic control model


Methods


```{r}
set.seed(500)
cols <- c('darkorange', 'darkgreen')
moz2_fortified <- moz2_fortified
map <- moz2_fortified %>%
  filter(id %in% c('Manhiça',
                   'Magude'))

centroids <- moz3
centroids <- centroids[centroids@data$NAME_2 %in% c('Manhiça', 'Magude'),]
centroids@data$color <- ifelse(centroids@data$NAME_2 == 'Magude',
                               'red', 
                               'white')
coords <- coordinates(centroids)
coords <- data.frame(coords)
names(coords) <- c('x', 'y')
centroids@data <- cbind(centroids@data, coords)
centroids$size <- rnorm(mean = 2, n = nrow(centroids))
centroids$size <- ifelse(centroids@data$NAME_3 == 'Magude',
                         0, 
                         centroids$size)
centroids$size <- centroids$size^10
ggplot() +
  geom_polygon(data = map,
               aes(x = long,
                   y = lat,
                   group = id,
                   fill = id),
               color = 'white',
               # color = 'grey',
               alpha = 0.8) +
  theme_map() +
  scale_fill_manual(name = '',
                    values = cols) +
  # theme_black(base_size = 0) +
  theme(axis.line = element_blank(), axis.text = element_blank(),
        axis.ticks = element_blank(), axis.title = element_blank(),
        legend.position = 'none') +
  geom_point(data = centroids@data,
             aes(x = x,
                 y = y,
                 size = size),
             alpha = 0.6,
             color = 'darkred')
```

***

- We employ the synthetic control method (SCM).  
- Weighted combination of potential control districts, the synthetic control, using the districts that best approximate the most relevant characteristics of malaria incidence in Magude before the intervention. 


### Magude vs. other districts

Results


```{r}

x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  group_by(date, place) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

ggplot(data = x,
       aes(x = date,
           y = p,
           group = place)) +
  geom_line(aes(color = place), alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Weekly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Weekly incidence: Magude vs. other districts',
       subtitle = 'All age groups') +
  theme_maltem()
```

***

Incidence, not adjusted for weighted contributions.

### Magude vs. other districts by age

Results


```{r}

x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  group_by(date, place, age_group) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

ggplot(data = x,
       aes(x = date,
           y = p,
           group = place)) +
  geom_line(aes(color = place), alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Weekly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Weekly incidence: Magude vs. other districts',
       subtitle = 'Segregated by age group') +
  theme_maltem() +
  facet_wrap(~age_group)
```

***

Effect is clearer in younger children.


### 0-4 year-olds: Magude vs synthetic control  

Results

 
img/synthetic1.png





### 5+ year-olds: Magude vs synthetic control  

Results
 
 
img/synthetic2.png

### Placebo testing for diarrhea

Results


img/diarrhea.png





### Effect of interevention

Results

img/ranjeeta_table.png




***

Relative to the synthetic control, the average treatment effect during the first post-intervention malaria season was a reduction in weekly incidence by 4 and 2 cases per 1,000 in those aged 0-4 and 5 or older, respectively. During peak malaria season (January - March), 1,939 cases were prevented in 2016, and 3,405 in 2017. 


### Our findings

Results



```{r}
x <- data_frame(year = 2015:2017,
                value = c(100, 100 - c(91.29, 81.3)))
ggplot(data = x,
       aes(x = year,
           y = value)) +
  geom_line() +
  geom_point(size = 20,
             alpha = 0.5) +
  theme_void() +
  geom_hline(yintercept = c(0, 100),
             lty = 2,
             alpha = 0.6) +
    geom_label(aes(label = round(value))) 
```

***

-  Across age groups during the peak malaria season (January-March) incidence declined by 91.29% and 81.3% in 2016 and 2017, respectively.  
- Has not achieved zero cases.  
- Unsure, at this point, to what extent we can attribute cases to importation or other causes. 

### Implications  


Discussion


- The strategy reduced malaria significantly.   
- Effectiveness quantified accurately enough to allow for granular cost-effectiveness.   
- Synthetic control method useful and generalizable for the estimation of similar initiatives’ effectiveness.   
- Policy-makers, researchers, and those actively engaged in or planning disease elimination initiatives, can benefit from both the methods and findings from this study.


### Thank you


Fin

[http://www.economicsofmalaria.com](economicsofmalaria.com)  
joe@databrew.cc  


### Extra slides  

Supplementary slides begin here.  


### The synthetic control

Results

```{r}
# Copy of Ranjeeta's table
x <- dplyr::data_frame(Indicator = c('Incidence, rainy season 2015',
                              'Incidence, rainy season 2014', 
                              'LLIN coverage',
                              'Pre-intervention precipitation',
                              'Pre-intervention temperature',
                              'Post-intervention precipitation'),
                `Magude 0-4` = c(3.65, 8.87, 24.82, 1.71, 24.52, 1.84),
                `Synthetic Magude 0-4` = c(5.1, 6.93, 24.77, 1.74, 24.51, 1.82),
                `Magude 5+` = c(4.02, 8.53, 24.83, 1.7, 24.52, 1.84),
                `Synthetic Magude 5+` = c(4.12, 6.29, 25.22, 1.71, 24.49, 1.83))
knitr::kable(x, format = 'html')

```

***


```{r}
uw <-
  df %>%
  filter(age_group == '0-4') %>%
  filter(disease == 'MALARIA',
         district != 'XAI-XAI CITY',
         district != 'MANHICA',
         district != 'MASSINGIR') %>%
  group_by(district) %>%
  tally
uw$w.weight <-
  c(0.00,
    0.22,
    0.00,
    0.19,
    0.01,
    0.00,
    0.00,
    0.27,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.07,
    0.00,
    0.00,
    0.24
    )
uw$age_group <- '0 to 4'

older <- uw
older$age_group <- '5+'
older$w.weight <- 
  c(0.00,
    0.00,
    0.00,
    0.00,
    0.10,
    0.00,
    0.00,
    0.26,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.35,
    0.29
)
uw <- bind_rows(uw, older)
# Overwrite with Ranjeeta's weights!
ggplot(data = uw,
       aes(x = district,
           y = w.weight)) +
  geom_bar(stat = 'identity',
           alpha = 0.6,
           fill = 'darkorange') +
  theme(axis.text.x = element_text(angle = 90)) +
  labs(x = 'District',
       y = 'Weight',
       title = 'Weights by district') +
  ggthemes::theme_hc() +
  facet_wrap(~age_group,
             nrow = 2)
```


### Comparison with other findings


Discussion

- Similar findings obtained in a cluster randomized controlled trial (RCT) estimating the effectiveness of MDA with DHA/PPQ in Zambia.

- In that study, while the number of malaria cases significantly declined after MDA in low-transmission areas, drop was not significant in areas of high transmission (ie, Magude).

- Also consistent with the predictions of a consensus study that put together 4 models focusing on the impact of MDA with a drug having similar characteristics as DHA/PPQ.  

- The latter predicted an immediate (but transient w/o other interventions) decline in malaria prevalence after MDA.  

### Quasi-experimental methods in similar contexts


Discussion


- Quasi-experimental approaches have been applied in other elimination campaigns.  

- Barofsky et al. used SCM as an approach for the identification of the impact of a malaria campaign in a district in Uganda.   
- These studies are from 50s-60s and look at economic indicators, such as primary school completion or households consumption.   
- To our knowledge, this is the first quasi-experimental evaluation of an ongoing elimination initiative using routine national surveillance data on health indicators.   

### Comparison with RCT


Discussion

- Randomized control trials (RCTs) are the gold standard for impact evaluations.   
- However, the implementation of RCTs may imply financial, practical and ethical challenges.  
- Furthermore, RCTs have debatable usefulness in identifying intervention feasibility and implementation bottlenecks, which are particularly important in disease elimination contexts.
- Quasi-experimental approaches are particularly useful when applied to routinely collected (administrative/observational) data. 

### Scaling up the method?


Discussion


- Given the focus over the last decade on malaria elimination with both large and small scale campaigns across the globe – quasi-experimental approaches offer a rigorous and inexpensive alternative to randomized-control trials. 

- Observational data is widely (and often publicly) available and true experimental approaches are often not feasible:  quasi-experimental approaches like the SCM are an excellent alternative for the evaluation of malaria elimination initiatives. 

- Approach is not only suitable for the estimation of the impact of the intervention in question, but also generalizable to malaria elimination campaigns globally.   

### Limitations 


Discussion


1. We rely on surveillance malaria cases reported at the health facility level, which is an incomplete and imperfect proxy for true malaria infection incidence since health-seeking behavior is especially low in low-income countries settings and may be different across geography and time.

2. The health facility registries from which we gathered data are themselves imperfect; we have no mechanisms by which to validate the entirety of the data. Still, BES data offers the best country-wide comparison of malaria trends at a granular level (weekly and by district).

3. Even with quasi-experimental data, we still can't fully control for abnormalities... (see next slide)

### 2017 has been anything but normal

Discussion

```{r}
x <- 
  df %>%
  group_by(year, month) %>%
  summarise(p = sum(cases) / sum(population) * 1000) %>%
  ungroup %>%
  mutate(year = factor(year))

cols <- colorRampPalette(RColorBrewer::brewer.pal(n = 9, 'Spectral'))(length(unique(x$year)))
ggplot(data = x,
       aes(x = month,
           y = p,
           group = year,
           color = year)) +
  geom_line() +
  geom_point() +
  scale_color_manual(name = 'Year',
                     values = cols) +
  labs(x = 'Month',
       y = 'Average incidence',
       title = 'Incidence by month, all districts',
       subtitle = 'Wide variability in monthly incidence') +
  theme_maltem()

```